Mean Field Theory based on BeliefNetworks for Approximate

Exact inference in large, densely connected belief networks is compu-tationally intractable, and approximate schemes are therefore of great importance. In the context of approximate inference in sigmoid belief networks, mean eld theory has received much interest. In this method the exact log-likelihood is bounded from below using a mean eld approximating distribution. In the standard mean eld theory, the approximating distribution is assumed to be factorial. In this paper we propose to use a (tractable) belief network as an approximating distribution. We show that belief networks t very well into mean eld theory, and no additional bounds are required. We derive mean eld equations which provide an eecient iterative algorithm to optimize the parameters of the approximating distribution. Simulation results on an inference problem indicates a considerable improvement over existing mean eld methods.